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ABSTRACT 

Intelligent tutoring systems are adaptive learning environments 
designed to support individualized instruction. The adaptation 
embedded within these systems is often guided by user models 
that represent one or more aspects of students’ domain 
knowledge, actions, or performance. The proposed project focuses 
on the development and testing of user models within the 
iSTART-2 intelligent tutoring system, which will be informed by 
dynamic methodologies and data mining techniques. My previous 
work has used post hoc dynamic methodologies to quantify 
optimal and in-optimal learning behaviors within the game-based 
system, iSTART-2. I plan to build upon this work by conducting 
dynamical analyses in real-time to inform the user models 
embedded within iSTART-2. I am seeking advice and feedback 
on the statistical methods and feature selection that should 
included within the new dynamic user model. The implications of 
this approach for both iSTART-2 and the EDM field are 
discussed. 
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1. INTRODUCTION 

Intelligent tutoring systems (ITSs) are adaptive learning 
environments that provide customized instruction based on 
students’ individual needs and abilities [1J. ITSs are typically 
more advanced than traditional computer-assisted training in that 
they adapt to the users’ performance and skill levels [2], The 
customizable nature of ITSs has resulted in the successful 
integration of these systems into a variety of settings [3-5J. 

One hypothesized explanation for the widespread success of these 
systems is that ITSs provide individualized feedback and adjust 
content based on the unique characteristics of each student or 
user. This pedagogical customization allows students to progress 
through learning tasks at a pace that is appropriate to their 
individual learning model [6], It also ensures that students are not 
only learning at a shallow procedural level, but they are gaining 
deeper knowledge at an appropriate pace. 

One way in which ITSs store and represent information about 
learners is via user models. User models embedded within ITSs 
incorporate detailed representations of learners’ knowledge, 
affect, and cognitive processes [7], It is important to note that 
these models are often continuously updating throughout the 
students’ interaction within the system. Thus, potentially, every 
student action or decision made within the system contributes to 
more accurate and holistic user models. Although this concept 
seems to be intuitive, researchers often struggle to determine what 
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information belongs within the models and how to optimally 
quantify the dynamic nature of that information. 

In prior work, my colleagues and 1 have proposed that dynamical 
systems theory and associated analysis techniques are useful tools 
for examining behavioral patterns and variations within ITSs 
L8,9]. Indeed, dynamic systems theory affords researchers a 
unique means of quantifying patterns that emerge from students' 
interactions and learning behaviors within an ITS. This approach 
treats time as a critical variable by focusing on the complex and 
fluid interactions that occur within a given environment rather 
than treating behavior as static (i.e., set or unchanging), as is 
customary in many statistical approaches. In the proposed work, I 
hypothesize that dynamical methodologies have strong potential 
to inform user models by quantifying changes in students’ 
interactions and learning behaviors across time. This 
quantification and modeling of behavior can inform decisions 
about how content and feedback should be presented to each 
student based on their current learning trajectory. The overall goal 
of the proposed work is to test the utility of real-time dynamic 
analyses as a way to inform user models about optimal (and non- 
optimal) learning behaviors within a game-based ITS. 

1.1 iSTART-2 

Interactive Strategy Training for Active Reading and Thinking-2 
(iSTART-2) is a game-based ITS designed to improve high school 
students’ reading comprehension via self-explanation strategies 
[10]. In previous studies, iSTART-2, and its predecessors, have 
been shown to be effective at improving students’ self-explanation 
quality and reading comprehension ability [11, 12]. 

iSTART-2 consists of two phases: self-explanation training and 
game-based practice. During training, students watch a series of 
videos that introduce them to and provide examples of self- 
explanations strategies. After students view these videos, they 
transition to practice (see Figure 1 for a screenshot of the game- 
based practice interface). During practice, students are able to 
interact with a suite of mini-games, personalizable features, and 
achievement screens [13]. The game-based practice embedded 
within iSTART-2 is designed to promote the generation and 
identification of self-explanation strategies. Within these practice 
games students are exposed to game mechanics that serve as a 
form of feedback on their understanding of the self-explanation 
strategies (see [II] for more details). 

The interface of iSTART-2 uniquely affords students substantial 
agency and control over their learning path by allowing them to 
choose how they engage with the practice environment [9]. Such 
freedom also affords researchers with the opportunity to explore 
and model how and when students engage with these features and 
activities, and to explore the implications of such choices (i.e., 
how they affect performance and learning). 
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1.2 Current Work 

My doctoral research will use educational data mining methods to 
inform and build dynamic student models within game-based ITSs 
such as iSTART-2 [13]. Specifically, this study will explore how 
dynamic techniques such as Hurst exponents and Entropy analysis 
can be used in real-time to quantify students’ behaviors, 
performance, and cognition while they learn within iSTART-2. 
Analyses of students’ logged choices have been shown to be a 
blueprint regarding successful and unsuccessful behaviors for 
learning [13, 15]. Therefore, the logged information from 
iSTART-2 will be used in conjunction with dynamical analysis 
techniques as a means to quantify various types of in-system 
behaviors and their impact on learning outcomes in real-time. This 
information will then be used to adapt the pedagogical content 
students are exposed to. 

2. Proposed Contributions of Current Work 

The current work has both local and global implications. Locally, 
the development of dynamic user models will improve iSTART-2 
pedagogy. Currently, iSTART-2 has limited user models (only 
guides self-explanation feedback) embedded within the system. 
Thus, the inclusion of a dynamic user model is expected to 
improve system feedback and guide the content presentation 
provided to students. For instance, one research question that 
arises from this work is how to support optimal learning 
trajectories for every student. Dynamic user models have the 
potential to recognize non-optimal learning behaviors and provide 
feedback or navigate students toward more effective learning 
behaviors within the practice environment. Thus, it is 
hypothesized that the implementation of dynamic models will 
improve the design and generalizability of iSTART-2. 

Globally, this project will contribute to the AIED and EDM fields. 
User models are an important and often crucial aspect of ITS 
development. However, very few systems (if any) use dynamic 
data mining techniques to inform their student models. This work 
will be among the first studies to use techniques such as Hurst 
exponent analysis in real-time to inform user models that will 
ultimately be used to adapt the content and feedback presented to 
students. The methods presented here are generalizable and thus 
can be used in a variety of settings beyond iSTART-2. Although 
the goal of the current work is to design user models for the 
iSTART-2 system, this work is driven by the overarching goal of 
gaining a better understanding of students’ learning processes. 

3. Previous Work 

My previous research has revealed that dynamic methodologies 
are useful tools for quantifying students’ behavioral patterns 
within iSTART-2 [8,9,13,14,15]. For instance. Entropy is a 
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dynamical methodology used to measure the amount of 
predictability within a system or time series [16]. My colleagues 
and I have employed post hoc Entropy analysis to quantify 
variations in students’ behaviors within iSTART-2 and related 
them to performance differences. Based on students’ choices 
within games, an Entropy score can be calculated that is indicative 
of the degree to which students’ choice patterns are controlled 
versus random. In [13], students’ Entropy scores were included 
within a regression analysis to examine how students’ choices 
within the system influenced their self-explanation performance. 
Students who engaged in more controlled interaction patterns (i.e., 
strategic and planned out) within iSTART-2 also generated higher 
quality self-explanations compared to students who acted in more 
random or impulsive manners. 

While Entropy provides an overall view of students’ choice 
patterns within a system, it does not capture fine-grained 
fluctuations that manifest over time. To address this issue, Hurst 
exponents have been conducted using iSTART-2 log data. Hurst 
exponents [17] are similar to Entropy analyses in that they 
quantify tendencies or fluctuations present within a time series. 
However, Hurst exponents also act as long-term correlations that 
can characterize the fluctuations that manifest across time. Hurst 
exponents classify these fluctuations as persistent, random, or 
antipersistent [18]. Using this approach, we can identify when 
students choose to perform the same action(s) repetitively [8], 
This technique affords a fine-grained look at students’ behaviors 
across time. Although Entropy and Hurst exponent analyses have 
shed light upon the effects of students’ interactions within an ITS 
on learning, the analyses thus far have all been conducted post hoc 
(i.e., using data mining techniques). Thus, the current work seeks 
to build upon these dynamical analyses and apply dynamic data 
mining techniques in real-time as a means to inform student 
models within iSTART-2. 

4. Advice Sought 

For this doctoral consortium, advice is sought regarding two core 
concerns. First, what features should be included in dynamic user 
models? Currently, I have solely focused on students’ behaviors 
and in-system performance within the game-based practice 
portion of the system. However, iSTART-2 has powerful logging 
functionality capable of collecting everything from mouse 
movements to keystrokes. Thus, in this setting I would benefit 
from expert opinions or discussions concerning what features 
should (or could) be included within dynamic user models. 

Second, what other dynamic methodologies and tools are 
available and relevant to user modeling? Thus far, I have used 
random walks. Entropy and Hurst analyses. However each of 
these measures have one or more weaknesses. For instance, to 
reliably calculate a Hurst exponent, multiple data points are 
needed (e.g., over 100), therefore calculating Hurst in real-time 
may not be practical in all situations (i. e., a single session study). 
Thus, I would benefit from expert opinion and guidance regarding 
other dynamic measures or methodologies that could be used in 
real-time as a way to inform user models within iSTART-2. 
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